feat: add graph runtime API by gongchensu · Pull Request #9 · InfiniTensor/InfiniRT

gongchensu · 2026-06-24T03:15:03Z

Summary

Adds generated C++ runtime declarations and dispatch support for stream capture and graph lifecycle APIs in scripts/generate_public_headers.py.
Adds native graph API coverage for CUDA-like backends in src/native/cuda/nvidia/runtime_.h, src/native/cuda/iluvatar/runtime_.h, src/native/cuda/metax/runtime_.h, and src/native/cuda/moore/runtime_.h.
Adds an adaptive graph runtime test in tests/test_native_graph.cc, wired through tests/CMakeLists.txt.

Motivation

InfiniRT should expose CUDA Runtime API-aligned graph capture and graph launch APIs through the generated C++ runtime API, without introducing a C API. This is needed by the standalone InfiniRT graph bridge work in InfiniCore.

Related: InfiniTensor/InfiniCore#1351

Type of Change

feat - new feature / new operator / new platform
fix - bug fix
perf - performance improvement (no behavioral change)
refactor - code restructuring without behavior change
test - adding or fixing tests only
docs - documentation only
build / ci - build system or CI configuration
chore - tooling, formatting, or other non-code changes
Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

Smoke Test Result

# ssh nvidia, image: infinirt-verify:nvidia-latest
clang-format --dry-run --Werror include/infini/rt.h tests/test_native_graph.cc
cmake -B build \
  -DWITH_CPU=ON \
  -DWITH_NVIDIA=ON \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/workspace/infini-rt-prefix
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

100% tests passed, 0 tests failed out of 8
Passed tests include test_nvidia_graph and test_runtime_dispatch.

Test Results on Supported Platforms

Platform	Affected	Build / Smoke Result	Full Result / Notes
NVIDIA	Yes	smoke passed	`ctest` passed: 8/8
Iluvatar	Yes	not run	No Iluvatar validation in this round
MetaX	Yes	not run	No MetaX validation in this round; API currently returns unsupported for graph capture
Cambricon	No	N/A - not affected	N/A
Moore	Yes	not run	No Moore validation in this round; API currently returns unsupported for graph capture
Ascend	No	N/A - not affected	N/A

Full `pytest` output (optional)

N/A

Benchmark / Performance Impact

N/A. This PR adds runtime API surface and tests; it does not tune kernel performance.

Notes for Reviewers

This PR intentionally keeps InfiniRT as C++ API only. No C API is added.
The public runtime graph API is generated under infini::rt::runtime.
MetaX and Moore expose the graph entry points for API completeness but currently return unsupported until native graph support is available.

* feat!: align runtime API and add runtime dispatch (#11) * Align runtime API with generated wrappers * Add default runtime dispatch specialization * Refactor runtime dispatch namespace * Use Abseil status for runtime device API * Revert "Use Abseil status for runtime device API" This reverts commit a26ddff. * Address runtime dispatch review feedback * Keep runtime API list in generator * Add TensorView constructor guard test * Align runtime memcpy kind constants with CUDA API * Use CUDA-style runtime memcpy constants * Use CUDA-style runtime memcpy constants * Move TensorView tests back into core test * Remove standalone TensorView test target * Remove standalone TensorView test file * Use fully qualified runtime API names in README * style: format runtime dispatch test * feat: refactor InfiniCore CPU runtime to InfiniRT (#8) Co-authored-by: Jiacheng Huang <huangjiacheng0709@outlook.com> * feat: add platform-adaptive runtime tests (#15) * feat: add runtime backend API foundation (#14) --------- Co-authored-by: spike-zhu <74974704+spike-zhu@users.noreply.github.com>

Map the Ascend graph lifecycle to aclmdlRI capture and replay symbols loaded from AscendCL at runtime. Expose Ascend graph capture and replay through the generated C++ runtime API and link dl only for WITH_ASCEND builds.

Move graph runtime validation into the common Runtime contract and keep StreamCreate aligned with CUDA semantics. Hide Ascend RI symbol probing helpers, use the Runtime Error alias consistently, and drop unverified Moore/Metax graph placeholders.

gongchensu self-assigned this Jun 24, 2026

gongchensu requested a review from voltjia June 24, 2026 04:24

voltjia requested changes Jun 24, 2026

View reviewed changes

voltjia requested a review from Ziminli June 24, 2026 08:49

Ziminli requested changes Jun 24, 2026

View reviewed changes

Comment thread src/native/cuda/runtime_.h

gongchensu force-pushed the feat/graph-runtime-api branch 3 times, most recently from aae1676 to a4bac7f Compare June 25, 2026 09:13

voltjia requested changes Jun 26, 2026

View reviewed changes

Comment thread src/graph.h Outdated

Comment thread src/graph.h Outdated

Comment thread src/graph.h Outdated

gongchensu force-pushed the feat/graph-runtime-api branch from a4bac7f to b9c8bad Compare June 30, 2026 06:31

This was referenced Jul 3, 2026

feat: add runtime backend API foundation #14

Merged

fix: align graph runtime API with runtime namespace #13

Merged

voltjia force-pushed the feat/graph-runtime-api branch 2 times, most recently from 81df7b1 to 9f82a06 Compare July 3, 2026 08:54

gongchensu and others added 5 commits July 7, 2026 06:43

feat: add graph runtime api

0f53e62

feat: add graph c api

82a450d

refactor: use generated C++ graph runtime API

63413be

chore: drop unrelated gitignore change

3daf6cb

gongchensu force-pushed the feat/graph-runtime-api branch from 2fc8a16 to 3daf6cb Compare July 7, 2026 06:43

feat: add Ascend graph runtime support

6668bcb

Map the Ascend graph lifecycle to aclmdlRI capture and replay symbols loaded from AscendCL at runtime. Expose Ascend graph capture and replay through the generated C++ runtime API and link dl only for WITH_ASCEND builds.

gongchensu force-pushed the feat/graph-runtime-api branch from b8de364 to a0e5246 Compare July 7, 2026 07:09

voltjia requested changes Jul 7, 2026

View reviewed changes

gongchensu force-pushed the feat/graph-runtime-api branch from a0e5246 to 0eb12cb Compare July 8, 2026 03:16

voltjia marked this pull request as ready for review July 8, 2026 03:36

voltjia requested review from Ziminli and voltjia July 8, 2026 03:36

voltjia approved these changes Jul 8, 2026

View reviewed changes

voltjia changed the title ~~feat: add graph runtime api~~ feat: add graph runtime API Jul 8, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add graph runtime API#9

feat: add graph runtime API#9
gongchensu wants to merge 7 commits into
masterfrom
feat/graph-runtime-api

gongchensu commented Jun 24, 2026 •

edited by voltjia

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

gongchensu commented Jun 24, 2026 • edited by voltjia Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Type of Change

Platforms Affected

Smoke Test Result

Test Results on Supported Platforms

Benchmark / Performance Impact

Notes for Reviewers

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gongchensu commented Jun 24, 2026 •

edited by voltjia

Loading